Papers with sequence-to-sequence architecture
Abstractive Summarizers are Excellent Extractive Summarizers (2023.acl-short)
Copied to clipboard
| Challenge: | Abstractive summarization systems have traditionally been fragmented, limiting the benefits of compatible models. |
| Approach: | They propose three new inference algorithms using sequence-to-sequence architectures to model extractive summarization with an abstractive summmarization system. |
| Outcome: | The proposed algorithms outperform existing models on CNN and Dailymail and show that they are more efficient than existing models. |
EfficientOCR: An Extensible, Open-Source Package for Efficiently Digitizing World Knowledge (2023.emnlp-demo)
Copied to clipboard
| Challenge: | Existing OCR engines fail to provide accurate, cost-effective and sample-efficient character recognition for public domain documents. |
| Approach: | EffOCR is an open-source optical character recognition package that is accurate, cheap to deploy and sample efficient to customize to novel collections, languages, and character sets. |
| Outcome: | EffOCR model trains character retrieval problem and scales to novel collections, languages, and character sets. |
Learning Matching Models with Weak Supervision for Response Selection in Retrieval-based Chatbots (P18-2)
Copied to clipboard
| Challenge: | Existing methods to learn matching models for retrieval-based chatbots are lacking. |
| Approach: | They propose a method that uses a sequence-to-sequence architecture model as a weak annotator to judge the matching degree of unlabeled pairs and performs learning with both the weak signals and the unlabed data. |
| Outcome: | The proposed method improves on two public data sets on matching models on retrieval-based chatbots. |
Learning Simplifications for Specific Target Audiences (P18-2)
Copied to clipboard
| Challenge: | Text simplification is a monolingual text-to-text transformation task . data from TS data can contain multiple simplifications of the same original text . |
| Approach: | They propose to use sequence-to-sequence neural models to build models tailored for specific grade levels. |
| Outcome: | The proposed model outperforms state-of-the-art approaches for a monolingual text-to-text transformation task. |
NL2pSQL: Generating Pseudo-SQL Queries from Under-Specified Natural Language Questions (D19-1)
Copied to clipboard
| Challenge: | Existing studies focus on generating SQL codes from natural language questions . however, questions cover more diverse tasks including table manipulation or performance issues . |
| Approach: | They propose a task to generate pSQL codes from natural language questions . they define two new metrics suitable for the task, Canonical-BLEU and SQL-BLUE . |
| Outcome: | The proposed task generates well-formed queries on under-specified database issues. |
Disentangled Sequence to Sequence Learning for Compositional Generalization (2022.acl-long)
Copied to clipboard
| Challenge: | Existing models struggle to generalize to unseen compositions of seen components . a new approach allows for disentangled representations and better generalization . |
| Approach: | They propose an extension to sequence-to-sequence models which encourage disentanglement by re-encoding source input. |
| Outcome: | The proposed extension delivers better generalization and more disentangled representations . human expressions can be understood by combining known atomic components . |
Gated Transformer for Robust De-noised Sequence-to-Sequence Modelling (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Noisy texts are common in user-generated texts that appear abundant in social media platforms like SMS, online chat, email, blogs, wikis etc. |
| Approach: | They propose a sequence-to-sequence architecture that uses a gating mechanism to detect types of corrections required from English texts. |
| Outcome: | The proposed architecture performs better than non-gated models on machine translation and Summarization tasks. |
Do sequence-to-sequence VAEs learn global features of sentences? (2020.emnlp-main)
Copied to clipboard
| Challenge: | Autoregressive language models are often trained without explicit conditioning labels . authors question claim that latent vectors can capture global features in unsupervised manner . |
| Approach: | They propose to use a sequence-to-sequence architecture to learn latent variables . they find that VAEs are prone to memorizing the first words and sentence length . |
| Outcome: | The proposed model is prone to memorizing the first words and sentence length, the authors show . et al., 2016: a new model learns latent variables that are more global, more predictive of topic or topic labels . authors question this claim, say it is a waste of time and money . |
Improving Knowledge Graph Completion with Generative Hard Negative Mining (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for knowledge graph completion (KGC) use generative methods with a self-information-enhanced training strategy to generate high-quality negatives. |
| Approach: | They propose to leverage a sequence-to-sequence architecture to generate high-quality hard negatives from the same decoding distributions as the anchor. |
| Outcome: | The proposed method produces high-quality negatives with good hardness and diversity on three KGC benchmarks. |
Keyphrase Generation with Correlation Constraints (D18-1)
Copied to clipboard
| Challenge: | Existing methods for keyphrase generation ignore correlation among keyphrases, resulting in duplication and coverage issues. |
| Approach: | They propose a new sequence-to-sequence architecture for keyphrase generation that captures correlation among keyphrases by preceding phrases to eliminate duplicate phrases and improve result coherence. |
| Outcome: | The proposed model outperforms the state-of-the-art method on benchmark datasets in terms of accuracy and diversity. |
Efficient OCR for Building a Diverse Digital History (2024.acl-long)
Copied to clipboard
| Challenge: | Current optical character recognition (OCR) systems are poorly extensible to low-resource document collections, as learning a language-vision model requires extensive labeled sequences and compute. |
| Approach: | They propose to model optical character recognition as a character level image retrieval problem using a contrastively trained vision encoder. |
| Outcome: | The proposed model is more sample efficient and extensible than existing architectures, enabling accurate OCR in settings where existing solutions fail. |